Import necessary libraries for the experiment

Loading the necessary data files and data Preparation

CORRELATION BETWEEN DATA LABELS

The correlation figure below shows that Toxic" comments are clearly correlated with both "obscene" and "insult" comments. Interestingly, "toxic" and "severe_toxic" are only weakly correlated. While we can also observe that, "Obscene" comments and "insult" comments are also highly correlated, which makes perfect sense.

DATA PREPROCESSING

Deep Neural Networks input layers make use of input variables to feed the network for training the model. But in this task (experiment), we're dealing with words text. How do we represent these words in order to feed our model?

In our experiment, we used densed representation of those text (comments) and their semanticity together. The advantage of using this approach is the best way for fitting neural networks onto a text data (as in our case), as well as less memory usage compared to other sparse representation approaches.

Word Embedding

Two ways to feed embeddings to neural networks:

For the first embedding, we used keras preprocessing (Text Preprocessing) libraries. This class allows to vectorize a text corpus, by turning each text into either a sequence of integers (each integer being the index of a token in a dictionary) or into a vector where the coefficient for each token could be binary, based on word count, based on tf-idf

Load the FastText Word Embeddng Corpus

Convert the word embedding into tokenized vector

Prepare the embedding matrix vectors in order to feed/pass the neural network

Visualize the Orchestrated LSTM-fastText Model Architecture

4) Bidirecitional LSTM with fastText

Visualize the Orchestrated Bidirectional GRU+FastText Model Architecture

Score Confusion Table